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Analytic MMSE Bounds in Linear Dynamic 
Systems with Gaussian Mixture Noise Statistics 

Leila Pishdad, Fabrice Labeau 


Abstract —Using state-space representation, mobile object po¬ 
sitioning problems can be described as dynamic systems, with 
the state representing the unknown location and the observa¬ 
tions being the information gathered from the location sensors. 
For linear dynamic systems with Gaussian noise, the Kalman 
filter provides the Minimum Mean-Square Error (MMSE) state 
estimation by tracking the posterior. Hence, by approximating 
non-Gaussian noise distributions with Gaussian Mixtures (GM), 
a bank of Kalman filters or Gaussian Sum Filter (GSF), can 
provide the MMSE state estimation. However, the MMSE Itself 
is not analytically tractable. Moreover, the general analytic 
bounds proposed in the literature are not tractable for GM noise 
statistics. Hence, in this work, we evaluate the MMSE of linear 
dynamic systems with GM noise statistics and propose its analytic 
lower and upper bounds. We provide two analytic upper bounds 
which are the Mean-Square Errors (MSE) of Implementable 
filters, and we show that based on the shape of the GM noise 
distributions, the tighter upper bound can be selected. We also 
show that for highly multimodal GM noise distributions, the 
bounds and the MMSE converge. Simulation results support the 
validity of the proposed bounds and their behavior in limits. 

Index Terms —Minimum Mean-Square Error estimator, ana¬ 
lytic bounds on Minimum Mean-Square Error, Gaussian mixture 
noise, online Bayesian filtering, Gaussian sum filter 


I. Introduction 

Due to the growing use of location-aware applications and 
ubiquitous systems, there has been an increasing need for 
more precise and accurate positioning systems. For outdoor 
scenarios, GNSS technologies can provide approximations 
with errors of 1 to 5 meters Q, which is suitable for outdoor 
applications. However, due to the increased multipath fading 
and non-line-of-sight (NLOS) conditions of indoor environ¬ 
ments, indoor positioning and location tracking can be more 
challenging Q. Therefore, Bayesian tracking techniques have 
been widely used to improve the location estimation by track¬ 
ing the posterior, or belief function Q. Specihcally by using 
state-space representation for positioning problems, the motion 
and sensor models are dehned by the process and measurement 
equations, respectively. Using these and the available sensor 
measurements, Bayesian tracking techniques, can approximate 
the posterior, hence provide the MMSE state estimator, as it is 
the expected value of the posterior Q, Q. For linear systems 
with Gaussian process and measurement noise, the Kalman 
hlter can optimally track the posterior and provide 

the MMSE state estimator However, in many scenarios, 
neither the process, nor the measurement noise are Gaussian. 
For instance, in Q, a non-Gaussian trimodal process noise 
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distribution is used, corresponding to the three different states 
of motion; constant velocity, accelerating, and decelerating. 
Additionally, mutipath fading effects could result in multi¬ 
modal non-Gaussian measurement noise distributions 0-II3- 
Since any distribution can be approximated by a Gaussian 
Mixture (GM) as closely as desired Q and the estimation will 
be asymptotically unbiased Gaussian sum approximation 
is an attractive method for modeling non-Gaussian distribu¬ 
tions. Moreover, since GMs can be viewed as conditionally 
Gaussian distributions, they enable the analytic evaluation of 
the posterior. Consequently, they have been used for modeling 
the non-Gaussian noise distributions 0,0, d, 

1^, prior 0, likelihood d, and 


also the posterior pT| , p2[ , p()[-|[ _ 

For linear dynamic systems with GM noise statistics, a bank 
of Kalman filters, or Gaussian Sum Filter (GSF), can be used 
to track the posterior and provide the MMSE state estimate 0, 
|37|. However, the MMSE itself is not analytically tractable. 
Moreover, the general analytic bounds on MMSE including 
Posterior Cramer-Rao p8| , Bobrovsky-Zakai and Weiss- 
Weinstein d, ED become non-tractable for GM noise 
statistics. Thus, in this work we evaluate the MMSE and 
provide analytic lower and upper bounds that unlike p2) , 
El, do not require complex numerical approximations. This 
is particularly important, as the MMSE is the best achievable 
MSE of any estimator, and by having low complexity methods 
to evaluate its bounds, suitable hltering schemes for specific 
applications can be designed. This work can be considered an 
extension to El^ in which the authors propose analytic bounds 
for static systems with GM noise distributions. Contrarily, in 
this work we consider dynamic systems which are more ap¬ 
propriate for mobile object positioning and location tracking. 
Moreover, we propose two implementable filters which have 
MSEs acting as upper bounds for the MMSE. The choice 
between these two upper bounds depends on the shape of the 
GM noise parameters. Moreover, since they are the MSEs of 
implementable biters, they can be used instead of the MMSE 
biter for applications where the upper bound for the MMSE 
provides the desired precision. The system and noise models 
used in this work, are the same models that we used in | [45l , 
E6| . However, the purpose of this work is to provide analytic 
bounds on the MSE of the MMSE state estimator, rather than 
proposing new bitering or reduction schemes. 

The rest of this paper is organized as follows: In Section [n| 
we provide the details of the system model and introduce the 
notation used in this paper, as well as the details of GSF, 
which is the MMSE biter. In Section uni we evaluate the 


MMSE and provide its analytic lower bound in Section III-A 


In Section |III-B we propose two upper bounds corresponding 
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to the MSEs of two implementable filters. Depending on the 
shape of the GM noise models in the system, the tighter upper 
bound can be chosen. This is shown through simulations in 
Section IV Finally, the paper is concluded in Section |V] 


II. System Model 

Suppose a discrete-time linear dynamic system, described 
by the following dynamics and measurement equations: 


Xfe = FfcXfc_i-f Vfe, (1) 

Zfe=TffeXfe+Wfc, (2) 

where {xf;,/c G N}, and {zfc,fc G N} are the state and mea¬ 
surement sequences, respectively. The process noise, {v^, k G 
N} is an i.i.d. random vector sequence with the known pdf 
p(vfe). The measurement noise, {wfc,fc G N} is also an 
i.i.d. random vector with the known pdf p{wk), and it is 
independent from the process noise. The matrices Fk and Hk 
are known and they dehne the linear relationships between the 
current and previous state vectors, and the current state and 
measurement vectors, respectively. If we denote the size of the 
state vector by Ux and the size of the measurement vector by 
Uz, the process noise is of size Ux and the measurement noise 
is of size n^. The matrices and are of size nx x nx 
and Uz X rix, respectively. 

In many signal processing applications, it is required to esti¬ 
mate the current unobservable state of the system, x^, from its 
available noisy measurements, zi-k. In positioning problems, 
the unobservable state denotes the unknown location, and the 
measurements are the information gathered from the location 
sensors. Therefore, Q represents the motion model and Q 
denotes the sensor model. 

Since the MMSE estimator of state is the expected value of 
the posterior distribution in Bayesian tracking tech¬ 

niques the state is estimated probabilistically, by approximat¬ 
ing the posterior distribution, p(xfe|zi:fe). For the special case 
of Gaussian process and measurement noise distributions, the 
Kalman hlter optimally tracks the mean and covariance matrix 
of the Gaussian posterior Q, hence providing the MMSE state 
estimate. However, for non-Gaussian noise distributions other 
approximations need to be used. 

Gaussian Mixtures (GM) have been an attractive method 
for approximating non-Gaussian noise distributions, as they 
provide asymptotically unbiased estimations fT^ , with the 
desired level of accuracy §□ Moreover, by using the condi¬ 
tionally Gaussian representation for Gaussian Mixtures (GM), 
the MMSE state estimator can be evaluated by a bank of 
Kalman biters, or Gaussian Sum Filter (GSF) Q, | [J7) (see 
Section |II-A[ ). Therefore, in this work we assume that the 
process noise is estimated by a GM distributior0 with 
clusters, cluster means {u^, 1 < i }, cluster covari¬ 
ance matrices 1 < * < }, and mixing coefficients 

^The parameters can be chosen such that the integral of the approximation 
en'or over the sample space is as small as desired. 

^Expectation maximization (EM) algorithm can be used to approximate a 
distribution by GMs, for instance see 0 


i.e. 

p(vfc) « ^ (3) 

i=l 

where -^(x; represents a Gaussian 

distribution with argument x, mean fi, and covariance matrix 
E. Similarly, the measurement noise distribution can also be 
approximated as: 

p(wfe) « Rij, (4) 

where, is the number of clusters of the GM distribution 
with coefficients 1 < j and Yhj 'Pk — 1- The 

mean and covariance matrix of cluster j, 1 < j < Cw^. are b^. 
and respectively. 

A. MMSE Filter 

Using the pdf approximations in (|^-Q for the system de- 
bned in ([T]i the posterior can be viewed as having multiple 
models, ; 1 < i , 1 < J < |, each correspond¬ 

ing to a mode in the posterior, which is a unique combination 


of the clusters in Q and 0- Hence, we can write: 

p(Xfe|zi:fc) = {Zk)p(xk\zi-.k, (5) 

i,3 

which denotes a GM distribution with x clusters, 
with coefficients 

Mfc (Zfe) =p(^fc^|zi:fc)- (6) 

The mode-conditioned posterior, p(^k\'Z‘i-k, is a Gaus¬ 

sian distribution with the pdf, 

p(xfc|My,zi:fc) =Af(xfe;:r;^^,P;^|J, (7) 

and its parameters and can be evaluated using the 
mode-matched Kalman biter 0, 0, as follows: 

= Tl.Xfc_i|fc_i -f 111 , (8) 

Pi\k-i=Ql + FkPk-i\k-iFl, (9) 

= Hkx\\k-i+K^ (10) 

= HkPl\k-iHl F P^k^ ( 11 ) 

^'i=Pl\k-iHlSr\ ( 12 ) 

x^\k = + ^k (zfc - Zfc ) , (13) 

= (14) 


where -kk-i\k-i and Pk-i\k-i are the estimated state and the 
estimation error covariance matrix at fc — 1, respectively, and 
(.)^ indicates the transpose of its argument. The matrix 
and the vector \N]^ represent the measurement prediction co- 
variance matrix and the biter gain, respectively. Consequently, 
the posterior in 0 can be tracked by a bank of Cy^ x Cy,^ 
mode-matched Kalman biters 0, 0. Using 

p{zk\Ml^ ,zi,k-^ = Af(zk-,z^ ,Sl^y (15) 
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zi:fe-i) = yviri, 

(16) 

the coefficients (zk) are evaluated as: 


Mfe (zfc) = p(^Ml^\zk,zi,k-i^ 

(17) 

p(zk\Mlfzi,k-i^p(^M]f\zi,k-i^ 

(18) 

p{zk\zi,k-i) 

>Vfc'PfcA/'(zfc; z*fc^ 5*^) 

(19) 


Z,m 


Since the posterior density in is a Gaussian mixture, 
having the mode-conditioned state estimations and covariance 
matrices, we can find the MMSE state estimate, 

^MMSE A E,,JXfc|zi:fc} = ^(20) 

and the covariance matrix of the combined estimate, 
pMMSE 4 E,,{ (x. - xfMS') (x, - X»“=)" |z,x.} (21) 

= + ( 22 ) 

where Ex{ 5 (x)} is the expected value of the function g(x) 
with respect to the random variable x with pdf p(x) ||^, and 

^k\k - Eix^{X/i; |Zl;/j;}. 

III. Unconditional MSE Evaluation and Bounds 
Using an approach similar to p^ , in this section we 
evaluate the unconditional MSE of the MMSE filter, denoted 

by 4- 

Lemma 1. The unconditional MSE of a sequential state 
estimator, with the state estimation error covariance matrix 
Pk\k> is equal to 

y tr(Ai|fc)p(zfe|zi:fe-i)dzfc, (23) 

where tr(.) denotes the trace of its argument. 

Proof: See Appendi x [A| ■ 

Using Lemma [T] and ( |21[ ), we can write: 

el = tY{Mk), (24) 

where 


®fe|/c®I|fcP(Zfe|zi:fe-l)dZfc. 

Additionally, from ( [T^ , we have 

p(zfc|zi:fe_i) 


Mfc (z/c) =- 


Hence, is evaluated as 


Ml=Y,WlP 


p(zfe|zi:fc_l) 

X P*jf|^p(Zfc|Zl:fc-l)dZfe 


(29) 


(30) 


(31) 

(32) 


since is not a function of the measurements (see ([l4|)). 
Now, using ( |30| ), ( [T3| l, and ( [T4l l we can write: 

/ (*fc|fc-i+Wfe (zfc-Z*jf)) 

X (xfc|fe-i (zfc 

X ff(zk-,z^ ,Sl^'jdZk (33) 

= E {^l\k-i^l\k-J + K'Sl^Wf) . (34) 

hj 


To evaluate we use and write: 

Z,m 

= Y.Y.^k'PiWivr I Jf(zk;ztSl^) 

i,j l,m 

V(x.;2‘r,5[") ,„.T 

E>VOTA'(zi,;2;‘,SJ‘) *1* *1'' 


which cannot be evaluated in closed-form due to the GM pdf 
in the denominator. Additionally, since it cannot be written 
as an expected value, numerical methods used for evaluating 
expectations cannot be applied to this problem. Hence, in this 
work we propose bounds for the MMSE. 


Alfe =y p““®^p(zfe|zi:fe_i)dzfc. (25) 

To evaluate the MMSE, we find the matrix fAk- Using ( |22| ) 
we can write 

Aik =Ail + Aik — Aik, (26) 

where. 



A. Lower Bound for the MMSE 

The MMSE filter evaluates and tracks the mode-conditioned 
posteriors and combines them using ( |20l i to provide the state 
estimation. However, at each iteration only one of the models 
is active, corresponding to the process and measurement 
noise clusters in effect at that iteration. Consequently, if 
prior information about the active model is available, by only 
including the posterior cluster corresponding to this model in 
the estimation of the current state, the total MSE is minimized. 
This is due to the fact that Kalman filter is the optimal state 
estimator if the model parameters are selected correctly |[^, 
Q. Assuming that cluster i and j are the active clusters of 
the process and measurement noise, respectively, we denote 
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the current active model by . The Matched filter state 
estimation and covariance matrix can then be dehned as: 


Y* A p* A pd 

^k\k — ^k\k- 

(36) 

Hence, we hav^ 


pMMSE \ r>* 

^k 1 k — ^k 1 k' 

(38) 

Now if denotes the MSE of Matched hlter, using LemmafT] 

we can write 


ef =/tr(Pfc*|fc)p(zfc|zi:fc_i)dzfc. 

(39) 

Using (|25|), (|3^, and ([39]l, we have 


cl > tv{Ml). 

(40) 

To evaluate we dehne 


^k-J -Pfe|fcP(z/c|Zl:/c-l)dZfc 

(41) 

= j y^p(Mfc'^|zi:fc^P"^^|^p(zfc|Zl:fc-l)dZfc 

(42) 

= J (z/c)PL‘|fcP(zfe|zi:fc_i)dZfc 

i,j 

(43) 

=Y.^inK\k- 

i,3 

(44) 

Corollary 1. The lower bound for the MMSF, is the 
ditional MSF of Matched filter, i.e. 

uncon- 


(45) 

Alternatively, Corollary can be proved using an approach 
similar to p4). Specihcally, using the dehnitions in ([27li-([29ll, 
we have 

tT {Ml), tv (Ml) fir (Ml) > 0. 

(46) 

Additionally, 


tr(A^fc) -tr(A^^) > 0, 

(47) 

since it is the expected value of the spread of means. 


^^k{^k) (&;% - Xk\k) - ^k\k) ■ 

(48) 

Hence, we have 



(49) 

By applying ([32]l to (|49]l, (|45]l is proved. 

■ 


B. Upper Bound for the MMSE 

Since the Matched hlter is not practical]^ the model corre¬ 
sponding to the posterior cluster with the maximum weight, 
(zfc) can be selected as the active model. In this section, 
we show that this hlter provides an upper bound for the 
unconditional MSB of the MMSE hlter. 


Definition 1. Assuming a GSF where cluster ij in the poste¬ 
rior has the maximum weight, i.e. 

d-kM = ma.xp‘f^{zk), (50) 

l,m 


GSF-R is defined as the Kalman filter ij in GSF. In other 
words, GSF-R selects and uses the Kalman filter with the 
maximum weight in GSF. We also define 


Mf = Mlf xtu. = X 


^k\k 


1] 

fc|fc> 


pR A 
^k\k ^k\k^ 


( 51 ) 


to address the parameters of GSF-R, when ( |50| ) is true. It is 
worth noting that only if = M^, represents the 

true covariance matrix of this filter. Therefore, we define 


Cf\k = Ex;,{(xfe - (xfc - |zi:fc|. (52) 


Proposition 1. The unconditional MSF of GSF-R, acts as an 
upper bound for the MMSF. 

Proof: Using ( |50l l, will be selected as if z^ is 
in where 

TZ]^ = ^Zk:yimfij;l<i,l<Cy^,l<j,m< 

WlViN{zk-Mi,S]^)>WiV'^N{zk-,it.Si^)]- ( 53 ) 

Hence, we can write 

p[Mf = Ml^ |zi:fe) = (zfc), (54) 

where lyi(.) is the indicator function of set A. Thus, given 
the available measurements, the probability of selecting 
when the true active model is Mjj^* is 

= MlfMi^*\z,,k) (55) 

=l.^^j(zfc)/ifc™(zfe) (56) 

which is a function of the current observation and the param¬ 

eters of the GM posterior, i.e. the coefficients, cluster means, 

pj2 

and covariance matrices. Now, if denotes the MSE of GSE- 
R, using Lemma [T] we have 

Efc" =/tr(c'^|fe)p(zfc|zi:fc_i)dzfe, (57) 

which is equal to the trace of where 


^It is worth noting that there could exist another model, ^ 

such that 

-^fcTfella - 

where ||.||p indicates the p-norm of its argument. Hence, the MSE of the 
estimator will be lower than However, since the active model is 

, this estimator can be biased. 


=/ (C^^k) Pi^k\zi..k-i)dzk. (58) 

The covariance matrix of GSE-R can be written as 
Cf\k =lEx;,{xfcxJ - XfcX^^ - x^x[ -f x^x^^|zi,fc|. (59) 

^Due to the unavailability of prior information about the active mode. 
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The first term in ( |59l l, can be evaluated using ( |2T] l as 

Ex,{xfcxJ|Zl:fc} = (60) 

Now, by conditioning on the active model and the selected 
model by GSF-R and using (|56ll, we have 


Ex,{xfcX^^|zi:fc| 

i,j l^m 


Ex,{xfcX«^|M« = M*^M^^Zl 


i,j l^m 


k\k 


— ^k\k ^ ^ 


k\k ' 


:fc} 

(61) 

T 

(62) 

(63) 


Similarly, we can find 


Exfc{xfcxX|zi:fe} 

Finally, by conditioning on the selected model by GSF-R and 
using •a, we can find 


lE-y 


[xfcXfe^|Zl:fc I = 


:fc| — X/ 

,{x^xX|M,« = M*^zi:fc} (65) 


E-y 






Hence, can be written as 


Cf\k + Xk\kxlik - Xk\k X] 

ij' 

- H i7?,‘4zfc)®rife®iife+y] 


( 66 ) 


'k\k 

’^k\k^k\k 

(67) 


^pMMSE ^ I 


'k\k 


Xk\k X~! 

as we have 

X] '^yy (Zfc)^ 


'k\k 


i,3 


^7 


( 68 ) 


'^l^iAZk)x 


^3 

k\k 


Thus, 


^7 


pMMSE ^ 

^k \k — ^ fc I fe ’ 


(69) 


(70) 


and using ( |24| l, ( |25| l, ( |57| l we have 

el<tiiMf).m (71) 

The details of evaluating Ai^, are provided in Appendix [ b| 


C. Alternative Upper Bound 

Kalman filter provides the MMSE state estimation for Gaus¬ 
sian noise distributions by tracking the sufficient statistics of 
the Gaussian posterior. For non-Gaussian posteriors, however, 
Kalman filter can only provide the Linear MMSE (LMMSE) 
state estimation Q. In this section, using an approach similar 
to 0, we use the Linear MMSE (LMMSE), to find an upper 
bound for MMSE. 

To apply Kalman filter to the system defined in ([T])-(|^, we 
need to use the moment-matched Gaussian distributions of the 
GM noise pdfs in ([^-Q. Hence, we define; 

= (72) 

^ 3 

Qk^J2^k (Qfc + - O ("fc - ’ (73) 

i 

(7?i + (bfc - bi) (b, - K.). (74) 

j ^ 2 

Thus, the LMMSE can be evaluated by running a Kalman filter 
on a Gaussian process noise with mean Ufc, and covariance 
matrix Qk, and a Gaussian measurement noise with mean 
and covariance matrix Rk, as follows: 

P^\k-l =PkPk-l\k-lPk +Qk, (75) 

Sf ^HkP^\k-iHl + Rk, (76) 

Pk\k ^Pk\k-i - Pk\k-iHlSf-^HkP^^k-i- (77) 

Proposition 2. The MSB of LMMSE filter, acts as an upper 
bound for MMSE, i.e. 

4 <tr(pX)- ( 78 ) 

Proof: Since the MMSE filter minimizes the MSE, any 
implementable filter will have an MSE greater or equal to the 
MSE of the MMSE filter. ■ 


Corollary 2. Upper bound of MMSE is the smaller of LMMSE 
and the MSE of GSF-R, i.e. 

<min{tr(M^),tr(p,%)}. (79) 

The shape of the GM noise parameters can affect the 
choice of the upper bound. Specifically, increasing the distance 
between the means of clusters in the GM noise distributions, 
increases the distance between the clusters of the GM pos¬ 
terior distribution. Consequently, tr^^PX) ^ larger 


value. This is supported through simulations in Section IV 


However, this trend is not valid for hence it is a 

better upper bound for highly multimodal GM posteriors (See 
Proposition |^. 


Proposition 3. Increasing the Mahalanobis distance between 
the clusters of the GM posterior decreases the difference 
between the bounds in 0, 0 and the MMSE. In the limit, 
the upper bound in 0 and the MMSE, converge to the lower 
bound in 0. 


Proof: Larger Mahalanobis distance among the clusters 
of the GM posterior, leads to increased difference between the 
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likelihoods, jA/" {^k\ ; 1 < * < Cy ^, 1 < j < Cyj^ |. 

In the limit, we have only one mixand corresponding to 
for which N ,SV!*'\ = 1, thus n]^*(zk) = 1. 


TABLE I 

The parameters of the GM models used for generating 

SYNTHETIC DATA 


Therefore, 



Model 1 

w = 10.2,0.2,0.2,0.2,0.2] 
m = c [-50,-30,0,30,50] 


yim ^ ij; Mfc"(zfc) = 0. 

(80) 

Model 2 

w = ]0.1, 0.1, 0.6, 0.1, 0.1] 
m = c [-50,-30,0,30,50] 

Hence, using ([80ll 

in (|28|), (|29|l, we have 


Model 3 

w = [0.5, 0.1, 0.1, 0.1, 0.2] 
m = c[-50,10,30,50,80] 


o' 

II 

1 

(81) 



which yields 

Q_ 

II 

(M 

(82) 

and they are denoted by w and m, respectively. The parameter 
c in this table is multiplied by the means to change the distance 
among the clusters of measurement noise distribution, i.e. the 




Additionally, since in the limit there is no overlap between the 
mixands of the GM posterior, from ( |55l l, we have 

VZm ^ ij] = 0. (83) 

Also, from ( [68] l 

(84) 

Using(|84ll in ( [8^ , we have 

ef =Y.^lrrPiTk-» 

l,m 


Fk = 


,Hk=[l 0], 


Vfc = tifc X [Atk, 1]"^, 


(85) 


IV. Simulation Results 

In this section we use synthetically generated data to 
approximate the MMSE and compare it with the proposed 
bounds, as well as the Posterior Cramer-Rao lower bound | [38l . 
In our simulation scenario, we assume a positioning system, 
with the following parameters: 

'1 Atk 
0 1 


( 86 ) 


where Atk is the time interval between the measurements Zk-i 
and Xfej^In our simulations, we use Atk = 0.1080 s for all 
iterations, k. 

The process model used in our simulations, is a random 
walk velocity motion model. Therefore, in ([T]) we have: 


(87) 


where Vk is a univariate random variable. In our simulations, 
we use the same modej^to generate process and measurement 
noise samples. To ensure that our simulation results are not 
model dependent, we used three different GM models to 
generate our data: 

Model 1: Symmetric distribution with the mixands all hav¬ 
ing the same weights. 

Model 2; Symmetric distribution with the mixands having 
not necessarily equal weights. 

Model 3: Asymmetric distribution. 


separation among the clusters. For the process noise, we use 
c = 1, to keep the state vector, hence the error, bounded]^ 

Fig. [T}|^ show the MMSF versus the Kullback-Leibler (KL) 
divergence between the GM measurement noise distribution 
and its moment-matched Gaussian pdf for the three GM 
models usedj^ In addition to the proposed bounds we have 
also provided numerically approximated Posterior Cramer-Rao 
lower bound for the MMSF hlter, for comparison purposes. 
The values in these hgures are evaluated at approximately %95 
conhdence interval with 1000 Monte Carlo runs. As shown in 
Section |III-A| the lower bound for the MMSF is the MSF 
of the Matched filter, for which we have prior information 
about the noise clusters in effect. Since, the unconditional 
MSF of Matched filter is only a function of the coefficients and 
the covariance matrices of the clusters (see (|44|)), it remains 
constant throughout our simulations, as we are only varying 
the separation between the clusters by changing the param¬ 
eter c. This is not true for the Posterior Cramer-Rao lower 
bound (PCRLB). Specihcally, PCRLB follows the trend of the 
MMSF more closely when compared with our proposed lower 
bound. Consequently, for smaller values of KL divergence 
(approximately KL < 1), it provides a tighter lower bound 
on the MMSE compared with our proposed lower bound. 
However, for larger values of KL divergence (approximately 
KL > 1), PCRLB is smaller than the proposed lower bound. 
Moreover, it is worth noting that unlike the proposed lower 
bound, PCRLB cannot be evaluated analytically for GSF and 
requires numerical approximations. 

Unlike the proposed lower bound, the performance of the 
upper bounds depends on the separation between the clusters. 
Specifically, for smaller values of KL divergence, i.e. KL < 
1.2, KL < 1.5, KL < 1.6 for Model 1, 2, and 3, respectively, 
the Kalman filter provides a tighter upper bound (LMMSE) 
when compared with GSF-R. However, further increasing the 
separation between the clusters, hence the KL divergence, 
improves the performance of GSF-R as an upper bound. This 
is because with more separation between the clusters, the 
parameter will have a smaller value for ij ^ Im*. 


The coefficients of the mixands are chosen such that the noise 
processes are zero mean. In our simulations, we assumed GM 
distributions with 5 clusters, each having a variance of 1. The 
coefficients and the means of the clusters are given in Table |I] 

^We use scalar measurements. 

®But with different separations among the clusters. 


^As shown in j^, if the filter is not completely consistent with the 
underlying system model, the estimation en'or is a function of the state vector. 
Therefore, if the state vector is unbounded, the filter will be unstable. To 
avoid this, we kept the variance of the state vector constant and only varied 
the measurement noise distribution. 

®The KL divergence between the process noise distribution and its moment- 
matched Gaussian pdf for Model 1, 2, and 3 is 2.0352, 1.9980, and 2.6255, 
respectively. 
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Fig. 1. MSE for the synthetic data generated using Model 1 vs. KL divergence 
between the measurement noise pdf and its moment-matched Gaussian. 



Fig. 2. MSE for the synthetic data generated using Model 2 vs. KL divergence 
between the measurement noise pdf and its moment-matched Gaussian. 



Fig. 3. MSE for the synthetic data generated using Model 3 vs. KL divergence 
between the measurement noise pdf and its moment-matched Gaussian. 

Consequently, the part of which depends on the distance 
between clusters will be smaller. 

It is also worth noting that the MSE of GSF-R follows the 
same trend as the MMSE versus the KL divergence between 
the measurement noise and its moment-matched Gaussian pdf. 
This is due to the fact that the performance of both filters 
depends on the separation among the clusters, since they 
both depend on the parameters (z^) to estimate the state. 
Specifically, when the clusters are very close (approximately 
KL < 0.1), both MMSE and GSE-R cannot find the active 
models accurately as the likelihoods for all clusters are very 
close. Moreover, since the distribution is very close to a single 
Gaussian, the Kalman filter performs similar to the MMSE 
filter. However, further increasing the KL divergence, increases 
the MMSE until it reaches its maximum, at 0.5 < KL < 1. 


This is because at lower separation levels, i.e. 0.1 < KL < 0.5, 
the MMSE filter cannot differentiate between the active model 
and the other models using the coefficients. Increasing the KL 
divergence past the peak of MMSE (KL > 0.5), results in 
lower MMSE since with higher separation, the coefficients 
are better indicative of the active model. This is why the MSE 
of GSE-R also decreases at these KL divergence values. 

As mentioned earlier, to avoid having an unbounded error, 
in this work we used a constant process noise pdf. However, 
as shown in | |45] , at very large KL divergences, e.g. KL > 6 
for process and measurement noise, the MMSE converges to 
the MSE of Matched filter, i.e. the lower bound (see Propo¬ 
sition 1^. However, since the wrong selection of the active 
model can lead to unbounded error Q, GSE-R becomes highly 
unstable for large values of KL divergence, especially for 
Model 1 and 3 which have a more uniform weight distribution 
(among the clusters of the noise distributions), and rely more 
on the likelihood for the correct selection of the active model. 

Based on the simulation results provided in this section, the 
proposed analytic bounds for MMSE show good performance, 
with the maximum distance between the bounds and MMSE 
equal to 20 dB for lower bound and 10 dB for the upper 
bounds. Moreover, depending on the GM noise distributions 
in the system under study, a tighter upper bound can be 
selected. Eor instance if the fitted Gaussians provide better 
representations of the noise distributions than their components 
with maximum weights, LMMSE can provide a tighter bound. 
This is true at approximately KL < 1.5 in our simulations. 
This is particularly important since the proposed upper bounds 
are the MSEs of actual implementable filters. Hence, when the 
upper bound for the MMSE provides the desired estimation 
precision, the suitable upper bound filter (Kalman or GSE-R 
depending on the noise models) can be used instead of the 
MMSE filter to save computational resources. 

V. Conclusion 

Positioning problems can be represented using state-space 
systems, with the state of the system denoting the unknown 
location. Using this representation, Bayesian tracking tech¬ 
niques can be applied to the available noisy sensor measure¬ 
ments to estimate the location. Eor linear dynamic systems 
with Gaussian noise, the Kalman filter provides the MMSE 
state estimation. Thus, by approximating non-Gaussian noise 
distributions with Gaussian Mixtures (GM), a bank of Kalman 
filters or Gaussian Sum Filter (GSF) can be used to find 
the MMSE state estimation. However, the MMSE itself is 
not analytically tractable. Specifically, the proposed analytic 
bounds in the literature, including Posterior Cramer-Rao p8) , 
Bobrovsky-Zakai p9) , and Weiss-Weinstein ||4g, gg, do not 
have a closed-form for GM noise distributions. Hence in this 
work we first evaluate the MSE of GSF, and then propose 
analytically tractable lower and upper bounds for the MMSE. 

The proposed lower bound is the MSE of a filter which 
only selects and uses the Kalman filter in GSF corresponding 
to the active model, by using prior information. The upper 
bounds, however, are given by filters that do not require prior 
information and therefore they are implementable: We propose 






























two filters with their MSEs acting as upper bounds for the 
MMSE. The hrst upper bound is the MSE of the Kalman 
hlter which is the Linear MMSE (LMMSE). The second upper 
bound, is the MSE of GSF-R which only selects and uses 
the Kalman hlter with the maximum weight in GSE. The 
choice between these two hlters depends on the GM noise 
parameters, as shown through simulations. We also show that 
in the limit, when the Mahalanobis distances between the 
clusters of the posterior approach inhnity, the MSE of GSE- 
R (upper bound) and the MMSE both converge to the lower 
bound. The fact that the upper bound hlters are implementable 
is particularly important, since for certain applications where 
the upper bound provides the desired accuracy, the suitable 
upper bound hlter can be used instead of the MMSE hlter, 
hence saving computational resources. 

Appendix A 
Proof of Lemma 1 

Proof: Assume a sequential state estimator, which pro¬ 
vides an estimate of the unknown state x^, using the available 
measurements zi-^. If we denote the state estimation by x*., 
and the state estimation error covariance matrix by Pk\k 7 the 
unconditional MSE of this estimator, ej., can be evaluated as 


]Exfc.Zi:fc{(xfc - Xfe)"^ (Xfc - Xfc)| 

(88) 

]Ezi^fc|lExfc|(xfe - Xfc)^ (Xfc - Xfc) Zkfcjj 

(89) 

J J i^k- Xfc)^ (Xfe - Xfc) 


X p(Xfc|Zl:fc)p(zi:fc)dXfcdZl:fc 

(90) 

tr(Pfe|fc)p(zfe|zi:fc_i)dzfc. ■ 

(91) 


Appendix B 
Evaluation of Ad™ 

In this section we provide the details on the evaluation of 

Ml. 

Using ( |58] l and ( |67] ), we have 
Ml=J + Xk\kxl^k (92) 

~ (93) 

i,3 

(94) 

i,3 

+ Kl^^k\k^k\k)pi^k\zi..k-i)dzk. (95) 
Hence, defining 

- Xk\k^t.j^^j{Zk)xl^Jp{Zk\2l:k-l)dZk, (96) 

id 

xF = I (97) 

id 


= / '^^TVf^d^k\k P^'^kW:k-l)dZk, (98) 

id 

and using ( |25| ), ( |2^ , and ( |2^ we can write 

Ml=Ml + Ml-Ml'-Mf+Mf. (99) 

To evaluate Ad^ , we can write: 

id 


X p(Zfc|zi:fe_i)dZfc ( 100 ) 

=E E I {^kdr, sd) 

id 1,'PrL 

^ ^dk^k\k^^^k (101) 

=E E ^kvr {K\k-i^i\k-J ( 102 ) 

id 1,'Pfi 

+K\k-i{^d-^iy^f (103) 

+ W'-(zf-zf)x^|,_/ ( 104 ) 


+Wi- (^ 5 '- + (z'r - z'D (zi- - Wf) , 

(105) 


where 

-im A J (zfc)zfcA/' (zfe; Zfc™, dzfc (106) 
Sd = J M (zk - zi’”) (zk - (107) 

xAA(zfc;z'™,4™)dZfc. (108) 

The integrals Ad^^ and Ad^^ can be evaluated in a similar 
manner. 
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